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Abstract 

We show algorithmic randomness versions of the two classical the- 
orems on subsequences of normal numbers. One is Kamae- Weiss the- 
orem (Kamae 1973) on normal numbers, which characterize the se- 
lection function that preserves normal numbers. Another one is the 
Steinhaus (1922) theorem on normal numbers, which characterize the 
normality from their subsequences. In van Lambalgen (1987), an al- 
gorithmic analogy to Kamae- Weiss theorem is conjectured in terms of 
algorithmic randomness and complexity. In this paper we consider two 
types of algorithmic random sequence; one is ML-random sequences 
and the other one is the set of sequences that have maximal complexity 
rate. Then we show algorithmic randomness versions of corresponding 
theorems to the above classical results. 
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1 Introduction 



Von Mises [T7] seemed to try to construct a probability theory that depends 
on given sample sequence but does not assume probability model a priori. In 
other words von Mises tried to construct a probability theory from a statisti- 
cal point of view. In order to achieve this program, he introduced the notion 
of collective (random numbers) and demand that its subsequences (selected 
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with place-selection rule) have the same frequency of symbols of the original 
sequence. Since then many authors studied the properties of subsequences of 
random numbers, e.g., Wald (1937), Church [3], Ville (1939). In this paper 
we show algorithmic randomness versions of the two classical theorems on 
subsequences of normal numbers. One is Kamae- Weiss (KW) theorem on 
normal numbers [6], which characterize the selection function that preserves 
normal numbers. Another one is the Steinhaus theorem on normal num- 
bers [13], which characterize the normality from their subsequences. In van 
Lambalgen [16], an algorithmic analogy to KW theorem is conjectured in 
terms of algorithmic randomness and complexity [21 [HJ [TUJ, [T2] . In this paper 
we consider two types of algorithmic random sequence; one is Martin-L6f 
(ML)-random sequences and the other one is the set of sequences that have 
maximal complexity rate. Then we show algorithmic randomness versions of 
corresponding theorems to the above classical results. 

Let Q be the set of infinite binary sequences. For x, y G Q, let x = x\x 2 ■ ■ ■ 
and y = yiy 2 ■ • • , Vz Xi, y- t G {0, 1}. Let r : N — > N be a strictly increasing 
function such that {i | yi — 1} = {t(1) < r(2) < • • ■ }, where N is the set of 
natural numbers. If £\ yi = n then r(j) is defined for 1 < j '< n. For i,i/GO 
let x/y be the subsequence of x selected at yi = 1, i.e., x/y = x T ^x T ^) • • •■ 
For example, if x = 0011 •• • , y = 0101 •• • then r(l) = 2,r(2) = 4 and 
x/y = 01 • • •. For finite binary strings x" := X\ ■ ■ -x n and := y\ ■ ■ -y n , 
x i/l/i is defined similarly. Let S be the set of finite binary strings and \s\ 
be the length of s G S. For s G S let A(s) := {scj|u; G where su is the 
concatenation of s and u. Let (Q, B, P) be a probability space, where B is 
the sigma-algebra generated by A(s), s G S. We write P(s) := P(A(s)). 

A probability P on Q is called computable if there is a computable func- 
tion A such that Ws,k \P(s)-A(s,k)\ < 1/k. For Ac S, let A := U seA A(s). 
A recursively enumerable (r.e.) set U C N x S is called (ML) test with re- 
spect to P if 1) U is r.e., 2) U n+ \ C U n for all n, where U n = {s : (n, s) G U}, 
and 3) P{U n ) < 2~ n . A test U is called universal if for any other test V, 
there is a constant c such that Vn V n+C C U n . In [10], it is shown that a 
universal test U exists if P is computable and the set 1Z P := {p^ =1 U n ) c is 
called the set of ML-random sequences with respect to P. 

Next, we introduce another notion of randomness. We say that y has 
maximal complexity rate with respect to P if 

lim -K{y n l )= lim —logPfo?), (1) 

n— >oo fl n— ¥od Tl 
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i.e., both sides exist and are equal. For example, y has maximal complexity 
rate with respect to the uniform measure (i.e., P(s) = 2~' s ' for all s) if 
lim^oo K(yi)/n = 1. If y is ML-random sequences with respect to a 
computable ergodic P then from upcrossing inequality for the Shannon- 
McMillan-Breiman theorem j5], the right-hand-side of (PQ) exists (see also 
[T%] ) and from Levin-Schnorr theorem (see (T2J) below), we see that ([Q) holds 
i.e., y has maximal complexity rate w.r.t. P. 

2 Algorithmic version of Kamae- Weiss theo- 
rem 

In Kamae [BJ, it is shown that the following two statements are equivalent 
under the assumption that liminf - Y^=i Vi > 0- 

Theorem 1 (Kamae- Weiss) 

(i) h(y) = 0. 

(ii) Vx x e Af -)• x/y e N, 

where h(y) is Kamae entropy fl\ Ub^ and Af is the set of binary normal 
numbers. 

A probability p on Q is called cluster point if there is a sequence {rij} 
Ws e S p(s) = lim #{1 < j < ^ | xj ■ ••x j+ \ 8 \- 1 = s}/rii. 

i—too 

From the definition, the cluster points are stationary measures. Let V(x) be 
the set of cluster points defined from x. From a diagonal argument we see 
that V(x) for all x. Kamae entropy is defined by 

h(x) = sup{/i(p) | p G V(x)}, 

where h(p) is the measure theoretic entropy of p. If h(x) = 0, it is called 
completely deterministic, see [BJ [13 [20] • The part (i)=^ (ii) is appeared in 

As a natural analogy, the following equivalence (algorithmic randomness 
version of Kamae's theorem) under a suitable restriction on y is conjectured 
in van Lambalgen [IB] . 

Conjecture 1 (Lambalgen) 

(%) \im. n ^ 00 K{y^)/n = 0. 
(ii) Vx xElZ^-x/yETZ 
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where K is the prefix Kolmogorov complexity and 1Z is the set of ML-random 
sequences with respect to the uniform measure (fair coin flipping), see [9]. 
Note that lim^oo K(y™)/n = h, P — a.s., for ergodic P and its entropy h, see 
PP. We show two algorithmic analogies to KW theorem (the following results 
are appeared in Takahashi p3] however we reproduce them for convenience). 
The first one is a ML-randomness analogy and the second one is a complexity 
rate analogy to KW theorem, respectively. 

Our first algorithmic analogy to the KW theorem is the following. 

Proposition 1 ( |14j ) Suppose that y is ML-random with respect to some 
computable probability P and Y^LiVi = 00 ■ Then the following two state- 
ments are equivalent: 
(i) y is computable. 
(11) Vx x eTZ^r x/y G Tl v , 

where TZ y is the set of ML-random sequences with respect to the uniform 
measure relative to y. 

Proof) (i)=>- (ii). Since YliLi Vi = 00 we have Vs \{x G f2 | s C x/y} = 2 - l s l, 
where A is the uniform measure. Let U be a universal test with respect to A 
and y(s) C S be a finite set such that {x G fl \ s C x/y} = y(s). Then y(s) 
is computable from y and s, and hence U v := {(n, a) \ a G y(s), s G U n } is a 
test if y is computable. We have x G U% -H- x/y G U n . (Intuitively U y is a 
universal test on subsequences selected by y). Then 

x G H ->■ a; ^ n„f/^ -H- x/y ^ n n f/„ -H- x/y G 11. 

Since 7/ is computable, W = 1Z and we have (ii). 

Conversely, suppose that y is a ML-random sequence with respect to a 
computable P and is not computable. From Levin-Schnorr theorem, we have 

Vn Km{y«) = - logP(^) + 0(1), (2) 

where i^m is the monotone complexity. Throughout the paper, the base of 
logarithm is 2. By applying arithmetic coding to P, there is a sequence z 
such that z is computable from y and y™ C l n = — logP(|/i ) + 0(1) 

for all n, where u is a monotone function and we write s C s' if s is a prefix 
of s' . Since y is not computable, we have lim n l n = 00. From (j2J), we see that 
Vn Km(z l { 1 ) = l n + 0(1). We show that if y G 7?. p then sup n l n+ i — l n < 00. 
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Observe that if y G 1Z P then \/n P(y") > and 

sup/ n+ i - l n < oo -H- sup - logP(y„ + i | y") < oo -H- inf P(y n+ i | y") > 

n n n 

hminf P(j/ n+ i | > 0. 

Let t/ n := {s | P(s | s[ shl ) < 2~ n }. Then P(U n ) < 2~ n and t/ := {(n,s) \ 
s G U n } is a r.e. set. Since y G limsup n £/„ liminf n P(y n+1 \ y™) = 0, we 
see that if y G 7£ p then sup n / n+ i — Z n < oo. (If U is r.e. and P{U n ) < 2~ n 
then 7l p C (limsup n ?7 n ) c , see [II].) Since Vn Km(z[ n ) = l n + 0(1) and 
sup n / n+ i — Z n < oo, we have Vn Km{z^) = n + 0(1) and z G 71. Since 2 is 
computable from y we have z/y ^ 7£ y . ■ 
Note that if ?/ is computable then y is a ML-random sequence with respect 
to a computable measure that has positive probability at y. 

Proposition 2 (| |14j ) Suppose that y has maximal complexity rate with re- 
spect to a computable probability and lim n - YH=i 2/i > . Then the following 
two statements are equivalent: 
(i) lim^^ \K{yl) = 0. 

(%%) Vx lim^oo \K(x"l) = 1 lim^oo ^ /y ^ K(x^ /y"|y") = 1. 

Proof) 
(i) =► (n) 

Let y := ?/i?/2 • • • G f2 such that ?/j = 1 if yi = and yi = else for all 2. Since 
\K{xl) - K{x n M)\ < K{y") + 0(1) and 

K(x^) = K(tf/y?,a»/jR\y?) + O0), 
if lim^oo K(y™)/n = and < lim n - J2™ =1 yi < 1 then we have 
lim K(x"l) /n = 1 



n— yoo 



lim -A-(x?/j/?,a:?M|j/r) = l 

n— >oo 77, 

lim -(K(x n JyM) + K{x n jrM)) = 1 

n— >oo 77, 

lim r*ljjr( x? / y? |,,») + n ~ ni 1 ifK/yfK) = 1 

n-voo flljj H U — U\ 

lim — #(z?/y?|y?) = 1 and lim — — K(x?/y?\y?) = 1. 

n->oo m n^oo n — U\ 
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where rii = \x™/y™\ = Yl^iVi- Similarly, if lim^oo K(y^)/n = and 

lim n , i Y!i=iVi = 1 tnen we have lim n^oo ^K(x^/y^\y^) = 1. 

(") =► (i) 
Suppose that 

lim -K(y?) = lim -- logP(^) > 0, (3) 

n— >oo fl n— >oo TL 

for a computable P. Let l n be the least integer greater than — log P(y™). 
Then by considering arithmetic coding, there is z = Z\Z% ■ ■ ■ G Q and a 
monotone function u such that C By considering optimal code for 

z l i we have Km(y™) < Km(z l { l )+0(1). From ([3]), we have lim n Km(y™)/l n = 
\im n Km(z[ n )/l n = 1. For l n <t< l n+1 , we have Km(z l 1 n )/l n+1 < Km(z[)/t < 
Km(z l { l+1 ) jl n . From ([3]), we have lim n l n+ i/l n = 1, and hence lim n Km(z^) /n = 
\im n K(z?)/n = 1. 

Since 1) z[ n is computable from 2) \im n l n /n > by ([3]), and 
3) lim n i E™ =1 ^ > 0, we have limsup^^ j^j^K(z^/yf\y^) < 1. ■ 

Example 1 Champernowne sequence satisfies the condition of the proposi- 
tion and (i) holds, however its Kamae-entropy is not zero. 

Example 2 If y is a Sturmian sequence generated by an irrational rotation 
model with a computable parameter [7J [15] then y satisfies the condition of 
the proposition and (i) holds. 



3 Algorithmic version of Steinhaus theorem 

In Steinhaus [13], it is shown that 

Theorem 2 (Steinhaus) x is normal number iff for all q G (0, 1), u q {y | 
x/y is normal number} = 1, where u q is the binary i.i.d. process with pa- 
rameter q, i.e., u q {l) = q, u q (0) = 1 — q. 

We have an algorithmic analogies for this result. 

Proposition 3 Let q G [0, 1]. The following two statements are equivalent: 

(i) x G TZ u "' q 

(ii) 3 computable y and x G U re [ 0i i]7\! Mr ' r x/y G lZ Uq ' q , 

where TZ u i> q is the set of ML-random sequences w.r.t. u q relative to q. 



6 



Proof) By considering the ML-test on the subseqences selected by y, (i)=>- (ii) 
follows. Conversely if x/y G 1Z Uq,q then x/y satisfies the law of large numbers, 
for example, see [9]. Thus q is uniquely determined (in fact computable) from 
x/y and we have (i). ■ 

Proposition 4 Let w be a computable probability such that 

(a) Vy G TZ W , lim n K(y n )/n = 0, (b) lim„ Y.i<i< n Vi/ n ex%sts f or V e K w , 

and (c)\/e> 03y G Flim^^j/i/n > 1 - e. 

Then the following two statements are equivalent. 

(i) \im n ^ oc ±-K(x n ) = l. 

(ii) lim^oo ] -J m K(x n /y n ) = lforyen w . 

Proof) (i) =>- (ii) follows from the part of (i)=r- (ii) of Proposition 2. 
(ii)=>- (i): Observe that 

\K(x n ,y n )-K(x n )\ <K(y n ) + 0(l), (4) 
\K{x\y n ) -K(x n /y n ,x n /y n ,y n )\ < 0(1), (5) 
\K(x n /y n ) - K(x n /y n ,x n /y n ,y n )\ < K(x n /y n ) + K{y n ) + 0(1), (6) 

where y is defined in the proof of Proposition [2J From the condition, we 
have Ve > By G U w 1 - e < lim n < 1. Then Ve > By G 

7^1imsup n ^-^K{x x /y n ) = Umsup n jgffij ^ m K{x n /r) < jh 
and limsup n , „V n , K(y n ) = 0, where the latter equality follows from the con- 
dition (a). From (ii), (g), (EJ, and ©, we have Ve > 3y G ft^l - ^ < 
liminf n ^ n l /yn ^ K(x n ) < limsup n j-±-^K{x n ) < 1 + Since we can choose 
e > arbitrary, we have (i). ■ 
For example, w = J P p dp, where P p is a probability derived from irrational 
rotation with parameter p, satisfies the condition of Prop. 2, see [T5j . 



4 Discussion 



Both proofs of Proposition 1 and 2 have similar structure, i.e., the part (i) 
—> (ii) are straightforward and in order to show the converse, we construct 
random sequences (in the sense of Proposition 1 and 2, respectively) by 
compression. 

We may say that Proposition 1 is a Martin-L6f randomness analogy and 
Proposition 2 is a complexity rate analogy to KW theorem, respectively. 
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These results neither prove nor disprove the conjecture of van Lambalgen. 
However Martin-L6f randomness and complexity rate randomness give dif- 
ferent classes of randomness, and a strange point of the conjecture is that it 
is described in terms of different notions of randomness. 

As stated above we proved our propositions by constructing random se- 
quences. In jl] pp.962, a different direction is studied, i.e., a sequence that 
is not predicted by MML with respect to finite order Markov processes is 
considered. Such a sequence is called red herring sequence jl] and considered 
to be a non-random sequence with respect to MML and finite order Markov 
processes, in the sense that MML cannot find a finite order Markov model 
for that sequence. 

Although Proposition E] is simple, it is considered to be a model selection 
problem from partially observed sequence, i.e., identify the model among 
i.i.d. model class from its subsequence. We can trivially extend the model 
class to the set of ergodic processes in Proposition |3l e.g., choose the selection 
function y such that lim n J2i<i< n yi/ n = 1- We need further development of 
this problem. 
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